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SUMMARY 


Studies  which  impose  no  limit  upon  the  number  of  responses  an  observer 
can  make  usually  find  high  false-positive  rates  *  The  present  study  examines 
observer  performance  when  limits  are  imposed.  Forty -two  bombardier- 
navigators  were  divided  into  three  target  choice  groups:  2G,  40,  and  no  limit. 
They  examined  a  moving  strip  of  side-looking  radar  imagery  rear-projected 
onto  a  14  by  14-inch  display  screen  at  a  scale  of  1:130,000.  The  displayed 
image  covered  a  25-mile  wide  strip  of  terrain  and  moved  a l  12.3  inches/minute, 
simulating  a  1320-knot  mission  lasting  27  minutes.  Result  show  that: 

1.  The  percentage  of  the  available  targets  that  are  detected  incr  eases  appre¬ 
ciably  and  significantly  with  each  increase  in  the  number  of  allowed  target 
choices.  However,  the  number  of  nontargets  mistaken  for  targets  also  in¬ 
creases  appreciably  and  significantly. 

2.  With  increase  in  number  of  allowed  target  choices,  the  number  of  ncntargets 
mistaken  for  targets  increases  at  a  faster  rate  than  the  increase  in  number  of 
detected  targets.  This  means  that  accuracy  decreases. 

3.  The  largest  decrease  in  accuracy  for  all  types  of  targets,  either  combined 
oi  separately,  occurs  in  going  from  the  40-uhoice  to  the  no-limit  choice  condi¬ 
tion.  Unlimited  freedom  to  respond  ruins  accuracy. 

4.  From  the  beginning  to  the  end  of  the  test  runs,  only  the  most  limited  (20- 
choices)  group  exhibits  any  trend  in  accuracy ;  For  this  group,  accuracy  in¬ 
creases  with  duration.  Also,  for  this  test  group,  there  is  a  positive  and  signif¬ 
icant,  but  not  high,  correlation  between  accuracy  and  number  of  choices  that 
have  been  made, 

5.  Along  the  displayed  flight  path  there  is  a  high,  positive,  and  statistically 
significant  correlation  between  rate  of  responding  (rights  plus  wrongs)  and 
frequency  of  available  targets,  and  between  number  of  targets  detected  and 
frequency  of  occurrence  of  targets. 

6.  Rates  of  responding  of  groups  with  different  cumbers  of  allowed  target 
choices  are  significantly  different.  However,  when  allowance  is  made  for 
fluctuation  in  frt  juenev  of  occurrence  of  targets  along  the  test  run,  these  rates 
exhibit  little  fluctuation.  This  relatively  constant  response  pace  is  attained 
eariy  .n  the  test  run. 
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7.  Subject  behavior  is  consistent  with  the  hypothesis  that  when  subjects  believe 
that  more  care  or  caution  in  target  selection  can  be  profitably  exercised,  they 
behave  accordingly. 

8.  The  probability  that  a  target  will  be  detected  varies  widely  with  the  type  of 
target.  However,  rate  of  increase  in  percentage  of  targets  detecteu  with  in¬ 
crease  in  number  of  permitted  choices  vai  ies  in  a  similar  way  for  the  four  types 
of  targets. 

9.  Accuracy  varies  widely  with  type  of  target.  However,  rate  of  decrease  of 
accuracy  with  increased  number  of  allowed  choices  is  about  the  same  for  the 
four  types  of  targets.  No  one  type  of  target  is  particularly  responsible  for  the 
large  increase  in  errors  when  more  targets  are  allowed. 

10.  All  test  groups  found  the  majority  of  the  targets  on  the  upr  er  half  of  the 
display. 

11.  The  average  distance  traveled  on  the  display  screen  by  targets  prior  to 
their  detection  does  not  vary  with  change  in  the  number  of  allowed  choices. 
Restrictions  upon  number  of  choices  neither  speeds  up  nor  slows  down  speed 
of  reaction  to  targets. 

ii.  The  increase  in  detections  with  increase  in  number  of  allowed  target  choices 
is  largely  due  to  targets  of  low  visibility.  When  no  limit  on  number  of  responses 
was  imposed,  many  more  (44^  additional  targets  not  reported  by  anyone  in  the 
most  limited  group  are  reported. 

This  study  demonstrates  that  limiting  the  number  of  targets  that  may  be 
chosen  can  greatly  reduce  the  number  of  nentargets  mistaken  as  targets,  al¬ 
though  the  reduction  is  achieved  at  a  cost  (understandably)  in  the  percentage  of 
actual  targets  detected.  Response  restrictions  could  make  useful  some  rapid 
reconnaissance  and  reconnaissance/strike  systems  heretofore  considered  to  be 
unfeasible  due  to  the  high  false-positive  rates  of  certain  types  of  prior  studies 
and  analyses.  Further  research  is  needed,  however,  on  the  utility-of-choice 
l sstrictions  versus  confidence  judgments. 
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SECTION  I 


INTRODUCTION 


Reconnaissance  systems  are  used  to  discover  potential  targets  and  to  gain 
new  information  on  known  targets.  Strike  systems  deliver  explosives  or  other 
material  upon  targets  to  damage,  destroy,  or  otherwise  reduce  the  military 
threat  that  they  represent.  Reconnaissance/strike  systems  combine  both  func¬ 
tions,  striking  new  targets  that  they  find  or  known  targets  which,  upon  inspec¬ 
tion,  are  found  to  require  such  action. 

The  number  of  targets  found  as  well  as  the  observer’s  confidence  and  ac¬ 
curacy  depend  upon  how  many  tasks  in  addition  to  target  seareh  that  he  must 
perform,  his  instructions,  motivation,  etc.,  as  well  as  on  the  nature,  numbers 
and  types  of  target  that  come  into  view  on  his  display.  Research  that  allows 
observers  an  unlimited  (or  at  least  unspecified)  number  of  target  choices, 
whether  or  not  he  reports  a  confidence  judgment  with  each  response,  will  not 
necessarily  lead  to  valid  conclusions  about  the  oehavior  of  operational  systems. 

Numerous  studies  have  been  reported  on  the  behav  tor  of  observers  search¬ 
ing  for  targets  of  opportunity.  In  some  studies,  observers  have  been  required 
to  state  how  confident  they  were  of  the  correctness  of  each  response,  and  a 
check  was  made  of  the  confidence  categories.  (Bate  and  Self,  1968  a,  b; 

Sadacca,  1960) .  In  all  of  these  studies,  a  significant  increase  in  accuracy 
(proportion  of  responses  correct)  resulted  from  accepting  as  targets  only 
those  responses  given  the  highest  confidence  level  rating  available  to  the  sub¬ 
ject.  Because  of  limited  weapon  loads  in  actual  flight  situations,  it  typically  would 
not  be  possible  to  strike  all  objects  that  appear  to  be  targets,  or  even  all  those 
at  the  highest  expressed  confidence  level,  Behavioi  of  an  observer  with  only  a 
few  target  choices  allowed  may  be  quite  different  from  his  behavior  in  an  un¬ 
limited  choice  situation.  For  these  reasons,  it  is  important  to  examine  the 
effects  of  such  limitaticoe  upon  observer  behavior. 

In  operational  reconnaissance/strike  system,  particularly  when  searching 
for  targets-of-opportunity,  i.e.,  new  targets,  the  radar  observer  does  not  know 
how  many  targets  he  will  encounter.  However,  he  may  know  how  long  he  will  be 
flying  ever  hostile  territory,  and  he  certainly  knows  how  many  strike  weapons 
he  has  on  his  aircraft.  In  the  present  study,  both  flight  time  (mission  duration) 
and  the  strip  of  territory  sensed  by  the  radar  and  depicted  upon  the  display  are 
held  constant.  The  number  of  allowed  target  choices  at  the  observer's  disposal 
(number  of  strike  weapons)  is  varied. 

The  present  study  examined  several  trends  in  performance  as  number  of 
allowed  target  choices  was  ^educed ,  The  results  are  intended  to  indicate  to 
what  extent,  if  any,  the  results  of  other  studies  not  using  limited  target  choices 
must  be  modified  when  applied  to  the  design  or  evaluation  of  systems. 
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At  the  start  of  the  study  it  was  hypothesized  that:  (1)  accuracy  will  in¬ 
crease  as  the  number  of  target  choices  assigned  to  the  observer  decreases, 

( 2)  cbservers  will  "pace"  themselves  so  as  to  utilize  all  or  nearly  all  of 
their  available  weapons.,  and  (3)  accuracy  will  vary  directly  with  the  accuracy 
of  pacing,  e.  g. ,  an  observer  will  be  reckless  near  the  beginning  of  a  run  with 
a  large  number  of  weapons  or  near  the  end  of  a  run  when  several  weapons  are 
unexpended.  If  he  is  well  ahead,  i.e. ,  has  a  long  time  to  go  and  has  only  a 
few  weapons  left,  he  will  be  extremely  cautious,  (4)  observers  who  have  a 
limited  number  of  allowed  choices  will  be  more  accurate  than  observers  told 
only  to  find  targets. 
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SECTION  n 


PROCEDURE 


EXPERIMENTAL  DESIGN 

Forty-two  test  subjects  were  randomly  divided  into  three  groups,  each  con¬ 
taining  14  subjects.  One  group  had  no  limit  placed  on  the  number  of  objects 
that  its  members  could  designate  as  targets;  members  of  the  second  group  could 
select  up  to  40  objects;  and  observers  in  the  third  group  were  limited  to  20 
objects.  The  experimental  design  was  a  single  factor  {number  of  allowed 
choices)  randomized-groups  design  with  each  subject  being  tested  only  once. 
Statistical  analysis  of  the  data  was  conducted  in  accordance  with  procedures 
applicable  to  a  mixed  model  experimental  design. 

The  number  of  target  choices  assigned  to  the  three  choice  groups  was 
selected  to  cover  a  wide  range  of  values  and  to  identify  any  important  perfor¬ 
mance  trends.  The  unlimited  choices  case  was  selected  to  represent  the  condi¬ 
tion  prevailing  in  most  previously  published  studies;  namely,  no  limit  on  the 
permitted  number  of  targets  that  subjects  could  designate.  To  make  the  task 
more  meaningful  for  the  subjects,  the  target  choice  limits  were  put  in  terms  of 
number  of  available  weapons  to  strike  detected  targets,  i.e. ,  ir.  the  20-choice 
case  they  were  told  there  were  only  20  weapons  available  to  strike  detected 
targets,  etc. 

EXPERIMENTAL  CONTROLS 

The  following  experimental  controls  were  maintained  throughout  the  experi¬ 
ment: 

1.  All  subjects  received  intensive  training  on  side-looking  radar.  For  a 
description  of  the  training  procedures  used  by  this  laboratory  on  side-looking 
radar,  see  VanAusda'l  and  Self  (1964). 

2.  All  subjects  received  written  instructions  at  the  beginning  of  the  test,  and 
each  was  given  a  few  minutes  practice  with  the  experimental  apparatus. 

3.  The  illumination  level  of  the  projected  radar  imagery,  rate  of  motion,  and 
projector  focus  were  verified  prior  to  each  test. 

APPARATUS 

Projection  Device:  A  moving  strip  of  high  resolution  side-looking  radar 
film  was  projected  onto  a  14  by  14-incn  rear-projection  screen  by  a  Model  100A 
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optical  projector  built  for  this  laboratory  by  the  Hughes  Aircraft  Company.  It 
contains  a  variable  speed  film  drive  mechanism  which  allows  a  wide  range  of 
aircraft  speeds  to  be  simulated.  However,  only  the  12.3  inches/minute  speed 
was  used. 

Response  Recording:  The  console  in  which  the  projector  was  housed  had 
a  response  panel  with  14  push-button  switches  located  to  the  right  of  the  viewing 
screen.  The  console  is  shown  in  lig  1.  The  subject  placed  the  tip  of  an  illumi¬ 
nated  stylus  upon  the  screen  image  of  each  target  and  depressed  the  appro¬ 
priate  switch  to  indicate  the  type  of  target.  If,  after  activating  the  target-type 
switch,  the  subject  decided  (before  pushing  the  record  switch)  that  he  had  made 
an  error,  he  couid  depress  an  "error"  erase  switch  which  canceled  the  target 
name  readout  data.  If  the  subject  believed  tisat  he  had  pushed  the  correct  target- 
type  switch,  he  depressed  the  "record"  switch,  activating  a  35  mm  data  camera 
located  over  his  right  shoulder.  This  camera  recorded  the  position  of  the 
stylus  on  the  screen  image,  and  the  target-type  readout  information.  The  read¬ 
out  information  was  displayed  to  the  data  camera  as  illuminated  digits  on  a  small 
screen  to  the  left  of  th-s  display  screen. 

Remaining  Flight  Time:  By  reference  to  an  illuminated  clock  located 
slightly  above  the  response  panel  on  the  right  of  the  viewing  screen,  the  subject 
kept  track  of  the  remaining  flight  time.  Simulated  flight  duration  (viewing 
time)  was  2?  minutes  for  all  conditions. 

Weapon  Count:  The  number  displayed  on  an  illuminated  digital  counter 
located  to  the  right  of  the  screen  decreased  by  one  each  time  an  object  was 
designated  by  the  observer  as  a  target.  When  the  number  of  responses  made 
by  the  subject  was  equal  to  the  permitted  number  of  target  choices,  the  counter 
read  zero,  and  no  more  responses  were  allowed.  Thus  the  counter  kept  the 
subject  informed  of  the  number  of  "strikes"  that  he  had  left.  It  was  his  weapon 
supply  record. 

The  same  5-inch-wide  strip  of  high  resolution  side-looking  radar  film, 
collected  by  an  A  PS-73  (XH-3)  radr.r  set,  was  used  for  all  subjects.  Figure  2 
is  an  example  of  the  general  type  of  radar  imagery  used  in  this  study;  however 
the  imagery  shown  in  figure  2  is  at  a  different  scale  than  that  actually  used  in 
the  study.  The  radar  imagery  displayed  to  t-  si  subjects  was  at  a  scale  of 
1:130,000  and  was  a  ground  swath  25  nautical  miles  wide.  The  radar  picture 
moved  from  the  top  to  the  bottom  of  the  display  screen  at  approximately  12.3 
inches  per  minute,  simulating  an  aircraft  speed  of  1320  knots. 

Assisted  by  Series  200  Navigation  Charts,  various  city  and  state  maps,  and 
ground  truth  information  from  other  sources,  the  investigators  searched  the  radar 
film.  Sixty -three  targets  were  judged  to  he  visible  e,i  the  display  screen  (Appen¬ 
dix  f) .  The  four  types  of  targets  that  the  subjects  were  asked  to  find  and  identify 
were:  (1)  airfields,  (2)  dams,  (3)  railroad  yards,  and  (4)  tank  farms  or 
petroleum  refineries. 
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Figure  1.  An  Observer  at  Work  at  the  Display  Console. 


D 


SUBJECTS  AND  TEST  INSTRUCTIONS 


The  subjects  were  42  navigator-bombardiers  from  the  USAF  Strategic  Air 
Command.  At  the  beginning  of  each  experimental  trial,  each  subject  received 
a  set  of  typewritten  instructions  (See  Appendix)  describing  the  situation  and  the 
task. 

There  were  three  sets  of  instructions,  but  two  were  alike  except  that  one 
said  "Your  aircraft  carries  20  bombs  (or  missiles)",  where  the  otbor  set  of 
instructions  stated  that  40  weapons  were  available.  The  third  set  of  instruc¬ 
tions  imposed  no  limits. 

MEASURES  OF  PERFORMANCE 

Target  Recognitions  and  False  Positive  Responses:  A  target  recognition 
was  recorded  each  time  a  subject  pointed  to  one  of  the  63  recognizable  targets 
and  depressed  the  correct  target-type  switch  and  record  switch.  A  false  posi¬ 
tive  was  recorded  each  time  a  subject  pointed  to  a  nontarget  object  and  depressed 
a  target-type  switch  and  the  record  switch. 

Number  of  Incorrect  Responses:  An  incorrect  response  was  recorded  when 
the  subject,  designated  one  of  the  63  targets,  but  assigned  the  wrong  target  name 
to  it.  Since  these  rnisclassificatirns  were  fairly  uniform  across  treatments, 
and  amounted  to  less  than  l%of  all  responses,  they  were  disregarded  in  the 
statistical  analyses. 

Screen  Travel:  Measurements  were  recorded  for  the  distance  traveled 
down  the  screen  by  targets  and  nontargets  prior  to  depressing  the  record  switch. 


section  n: 


RESULTS  AND  DISCUSSION 


NUMBER  OF  RESPONSES  TO  TARGETS  AND  TO  FALSE  POSITIVES 

The  number-  of  responses  to  real  targets  and  the  number  of  responses  to 
nontargets  mistaken  for  real  targets  for  individual  subjects  is  given  in  table  I. 

In  the  20-choice  group,*  13  of  the  14  subjects  used  all  20  allowed  choices,  and 
1  subject  used  only  19.  In  the  4G-ehoice  group,**  4  subjects  used  all  40,  1  used 
39.  G  used  38,  2  used  37,  and  1  used  29,  i.e.t  only  3  used  less  than  38  of  the 
40  allowed.  The  footnotes  explain  slight  discrepancies  between  the  above  data 
and  the  tables.  Clearly,  most  subjects  in  both  of  the  limited  choice  groups 
expended  all  of  their  supply  of  available  weapons  (or  allowed  choices) ,  and 
the  remaining  subjects  expended  almost  all  weapons. 

The  test  group  with  no  limit  on  number  of  allowed  target  choices  may  be 
regarded  as  a  control  group.  The  mean  number  of  choices  for  this  group  was 
76.6,  which  is  about  twice  the  mean  for  the  40-choice  group  (mean  37.7) .  The 
nature  of  this  increase  is  obvious  from  examination  of  figure  3.  This  figure 
shows  the  number  of  responses  to  targets,  the  number  of  responses  made  to 
nontargets  mistaken  for  targets,  and  the  sum  of  the  two  types  of  responses. 

While  there  is  a  small  increase  in  the  number  of  genuine  targets  detected,  there 
was  a  large  increase  in  the  number  of  nontargets  mistaken  for  targets,  i.e.,  in 
'.alse  positives.  The  result  of  this  large  increase  in  false-positive  responses 
upon  accuracy,  defined  as  the  proportion  of  target  designation  responses  that 
are  made  to  genuine  targets,  is  discussed  in  another  section  of  this  report. 

Bartlett’s  homogeneity  of  variance  test  (1934)  indicated  significant  hetero¬ 
geneity  of  variances  for  both  number  of  targets  detected  and  number  of  nontargets 
mistaken  for  targets.  Homogeneity  of  variances  was  achieved,  as  shown  by 
Bartlett’s  test,  when  a  logarithmic  transformation  was  applied  to  the  data. 

Analysis  of  variance  (see  tables  II  and  III)  performed  on  the  log-transformed 
data  for  number  of  targets  detected  and  for  number  of  nontargets  mistaken  for 
targets  yielded  _F  ratios  of  72.  98  and  145.4,  respectively.  Both  of  these  are 


*  In  this  group,  4  of  the  399  responses  made,  or  l%of  them,  were  unscorable 
due  to  overexposure  in  the  data  camera. 

**  Here,  2  of  the  530  responses  made,  or  0.38%of  them,  were  unscorable: 
one  due  to  overexposure,  and  one  due  to  the  subjects  head  blocking  ihe 
view  of  the  camera. 


statistically  significant  (P  .001) .  Analyses  of  variance  on  the  untransformed 
or  raw  data  resulted  in  the  same  finding,  yielding  _F  ratios  of  55.9  and  79. 9, 
respectively,  both  statistically  significant  ( P  .  001) . 

Numbers  of  reported  detections  and  numbers  of  false  positives  both  varied 
significantly  with  the  number  of  allowed  target  choices. 

Using  the  log-transformed  scores,  the  number  of  responses  made  to  real 
targets  in  the  three  test  groups  were  analyzed  by  Duncan's  New  Multiple  Range 
Test  (NMRT;  {Duncan  1955) .  Transformed  scores  for  the  number  of  false 
positive  responses  were  also  analyzed  in  the  same  way.  Duncan's  Test  showed 
that  the  average  numbers  of  targets  detected  by  the  three  groups  were  all  signif¬ 
icantly  different  from  each  other  ( P  .  05) .  The  average  number  of  false  posi¬ 
tives  were  also  significantly  different  from  each  other  (P  .05) . 

Duncan's  Test  and  an  examination  of  the  means  (averages)  indicated  that 
each  increase  in  number  of  allowed  target  choices  resulted  in  a  statistically 
significant  increase  in  both  number  of  targets  that  were  detected  and  number 
of  false  positives. 

PERCENTAGE  OF  AVAILABLE  TARGETS  DETECTED 

By  definition,  percentage  of  available  targets  detected  is  100  times  the 
number  of  targets  detected  divided  by  the  number  of  targets  available.  Thus, 
it  is  a  linear  transformation  of  number  of  targets  detected.  It  follows  that 
the  results  of  the  analysis  of  variance  for  numbers  detected  will  hold  for  per¬ 
centages  detected.  In  short,  with  every  increase  in  number  of  choices  allowed 
there  was  a  statistically  significant  increase  in  percentage  of  available  targets 
that  were  detected. 

Incidentally,  since  50  targets  were  actually  detectable  in  the  sense  that 
one  or  more  of  the  42  subjects  found  them,  numbers  detected  can  be  converted 
to  percent  detected  simpJy  by  multiplying  by  two.  Using  this  relationship  and 
the  mean  number  of  detections  in  table  I  reveals  that  the  average  percentage 
of  targets  detected  were  29  4,  46.6,  and  57.0  percent  for  the  20,  the  40  and 
the  no  limit  test  groups,  respectively. 

OVERALL  ACCURACY  OF  RESPONSES 

The  ratio  of  number  of  correct  responses  that  are  made,  i.e.,  the  propor¬ 
tion  of  responses  that  represents  detected  targets,  is  called  response  accuracy. 
Usually  it  is  abbreviated  simply  to  "accuracy''  and  is  expressed  in  decimal  form. 

Earlier,  it  was  noted  that  when  the  number  of  target  choices  allowed  in¬ 
creased,  the  •-•crease  ir.  the  numoer  of  faise  positives  exceeds  the  gain  in 
detected  targets  (see  fig  3)  .  Also,  when  no  limit  was  set  to  the  numb*  -  r\ 
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TABLE  n 


ANALYSIS  OF  VARIANCE  OF  NUMBER*  OF  TARGETS  DETECTED 


Source  of  Variance 

Mean  Sq  '.are 

d.f.  F 

Meaning 

Between  Groups 

.597 

2  72.98*** 

Number  of  targets 

detected  varies  sig- 

Within  Groups 

.160 

39 

nificantly  with  number 

of  choices  allowed. 

*  Logarithmically  transformed  data  was  used. 
***  Statistically  significant  at  the  .001  level. 


TABLE  IH 

ANALYSIS  OF  VARIANCE  OF  NUMBER*  OF  NONTARGETS 
THAT  WERE  MISTAKEN  FOR  TARGETS 


Sources  of  Variance  Mean  Square  d.f,  _F  Meaning 


Between  Groups  3.431  2  145***  Number  of  false  posi¬ 

tives  varies  signifi- 

Within  Groups  .  024  39  cantly  with  the  number 

of  choices  allowed. 


*  Logarithmically  transformed  data  was  used. 
***  Statistically  significant  at  the  .001  level. 
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Figure  3.  Average  Number  of  Responses  for  Groups  With 
Different  Numbers  of  Allowed  Target  Choices. 


choices  that  were  permitted,  the  number  of  false  positives  exceeded  the  number 
of  genuine  targets  that  were  detected.  Thus,  accuracy  decreased  when  more 
choices  were  allowed. 

The  individual  accuracy  scores  for  the  42  subjects  are  given  in  table  I.  The 
arithmetic  means,  which  are  at  the  bottom  of  the  tabis,  are  .748,  .621,  and  .385 
for  the  20-choice,  the  40-choice,  and  the  no  limit  test  groups,  respectively. 

In  figure  4  these  averages,  plotted  against  number  of  allowed  target  choices, 
show  that  accuracy  decreased  appreciably  when  subjects  were  allowed  more 
choices.  The  accuracy  of  the  no-limit  test  group  is  especially  low:  Accuracy 
is  only  about  half  as  high  for  this  group  as  it  is  for  the  two  limited-choice 
groups.  An  object  designated  as  a  target  by  the  no-iimit  group  is  twice  as 
likely  to  be  a  nontarget  as  it  is  to  be  a  target.  With  the  two  limited-choice 
groups,  a  designated  object  is  more  likely  to  be  a  target  than  to  be  a  nontarget 
object. 

Since  accuracy  scores  are  proportions,  an  arc  sine  transformation  )f  the 
scores  was  needed  to  secure  homogeneity  of  «*»  >'iance  so  that  the  data  could  be 
analyzed  by  analysis  of  variance.  This  analysis  is  summarized  in  table  IV. 

The  obtained  of  62.9  is  statistically  significant  at  the  .001  levei,  indicating 
that  the  number  of  available  weapons  (permitted  target  choices)  has  an  effect 
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Figure  4.  Average  Accuracy  With  Different 

Numbers  of  Allowed  Target  Choices. 

upon  accuracy  of  responses.  Table  V  summarizes  an  analysis  of  variance 
of  untransforined  scores  and  its  significant  (P  .001)  _F  also  leads  to  the  same 
conclusion.  Multiple  comparisons  of  the  response  accuracies  of  the  three 
choice  groups  by  means  of  Duncan’s  Test  (NMRT)  reveals  that  all  three  groups 
are  significantly  (P  .001)  different  from  each  other  in  accuracy.  Each  in¬ 
crease  in  number  of  allowed  target  choices  resulted  in  a  significant  decrease  in 
the  accuracy  of  responses. 

DISTRIBUTION  OF  RESPONSES  OVER  THE  TERRAIN 

Subjects  were  told  at  the  start  of  the  mission  (experimental  trial)  that 
they  would  be  in  simulated  flight  over  "enemy"  territory  for  27  minutes.  During 
the  mission  the  time  clock  and  response  counter  were  always  on  display.  How¬ 
ever,  the  territory  was  unfamiliar  to  the  subjects,  and  they  were  told  only  that 
the  strip  of  terrain  was  in  the  United  States.  Also,  they  did  not  know  how  many 
targets  might  appear  during  the  mission.  The  two  limited-choice  groups  were 
told  to  make  the  best  use  of  their  supply  of  weapons  (See  Instructions  in  the 
Appendix) . 

With  these  instructions  and  conditions,  several  strategies  are  possible. 

One  procedure  would  be  to  use  up  all  of  the  available  weapons  (choices)  as 
quickly  as  this  could  be  done  without  undue  waste  of  munitions.  Another  strategy 
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TABLE  IV 


ANALYSIS  OF  VARIANCE  OF  ACCURACY*  SCORES 


Source  of  Variance  Mean  Square  d.f.  F  Meaning 


Between  Groups 

1725.52 

O 

62. 9***  Accuracy  varied  sig¬ 
nificantly  with  number 

Within  Groups 

27.43 

39 

of  choices  allowed. 

*  Arc  sine  transformed  scores  were  used.  Accuracy  is  proportion  of  responses 
that  were  made  to  genuine  targets. 

**“  Statistically  significant  at  the  .  001  level. 


TABLE  V 

ANALYSIS  OF  VARIANCE  OF  ACCURACY  SCORES 
(USING  NO  TRANSFORMATION) 


Source  of  Variance  Mean  Square  d.f.  JF  Meaning 


Between  Groups 
Within  Groups 


.4765  2  70  ***  Accuracy  varied  signi¬ 

ficantly  with  number  of 

.0068  39  allowed  choices 


***  Statistically  significant  at  the  .  001  level. 
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would  be  io  be  very  cautious  early  in  the  simulated  mission  and  then  to  respond 
rapidly  (pile  up  responses)  and  almost  recklessly  near  the  end  of  the  mission 
in  an  attempt  to  unload  the  remaining  weapons.  Still  another  approach  would  be 
to  adopt  a  somewhat  uniform  rate  of  expenditure  (pace)  that  would  fluctuate 
with  the  frequency  of  occurrence  and  the  difficulty  or  obviousness  of  the  targets. 
Such  a  manner  of  responding  would  be  intended  to  spread  responses  over  the 
entire  mission.  Other  strategies  are  easily  conjectured. 

Since  all  subjects  responded  to  what  they  thought  were  targets,  it  is  instruc¬ 
tive  to  examine  all  responses  (responses  to  genuine  targets  plus  responses  to 
nontargets  mistaken  for  targets)  per  consecutive  fifth  of  the  displayed  terrain. 
The  data  is  given  in  table  VI,  and  is  plotted  in  figure  5.  The  curve  labeled 
"possible  targets"  plots  the  63  targets  that  were  judged,  prior  to  testing  ob¬ 
servers,  as  being  detectable,  while  the  curve  labeled  "available  targets"  plots 
the  50  targets  that  were  each  responded  to  by  one  or  more  of  the  entire  pool  of 
42  test  subjects.  These  two  curves  plot  numbers  of  available  targets  distri¬ 
buted  along  the  Flight  Path  rather  than  number  of  responses,  as  do  the  other 
curves. 


TABLE  VI 

AVERAGE  HUMBER  OF  RESPONSES*  PER  SUCCESSIVE 
FIFTH  OF  THE  TERRAIN  STRIP 


Fifth  of 

Terrain  Strip 

TEST  GROUP 

Number  of 

Available** 

Targets 

Number  of 

Possible*** 

Targets 

20 -Choices 
Mean  S.D. 

40-Choices 
Mean  S.D. 

No- Limit 
Mean  S.D. 

First 

6.86 

2.28 

10.29 

— 

20.79 

7.30 

14 

18 

Second 

2.71 

1.33 

5. 36 

12.36 

5.42 

6 

8 

Third 

4.29 

1.38 

6.86 

2.14 

14.00 

5.63 

9 

12 

Fourth 

5.07 

1.90 

10.86 

2.48 

18.93 

5.18 

16 

18 

Fifth 

0.71 

0.91 

4.36 

2.47 

10.57 

2.53 

5 

7 

*  Responses  include  both  detected  real  targets  and  false  positives. 

**  Available  in  that  one  or  more  of  the  42  subjects  made  a  response  to  the  target. 

***  Possible  in  that  this  number  of  targets  was  determined  to  exist  in  image  truth, 
i.e.,  were  judged  prior  to  the  experiment  tc  be  detectable. 
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*65  TARGETS  JUGGED,  PRIOR  TO  SUBJECT  TESTING,  AS 
BEING  DETECTABLE 

**  50  TARGETS  ’DETECTED'  BY  ONE  OR  MORE  OF  THE  ENTIRE  POOL 


Figure  5.  Average  Number  of  Responses  Per  Successive  Fifth  of  the  Strip 
of  Terrain  That  Was  Displayed,  Showing  the  Distribution  of 
Responses  Over  the  Duration  of  the  Simulated  Flight.  Responses 
Inch  de  Both  Target  Detections  and  Nontarget  Objects  Mistaken 
for  Targets. 


Examination  of  the  figure  reveals  that: 

1.  All  of  the  curves  on  the  graph  are  approximately  parallel  to  each  other. 
There  are  ro  intersections  o-  crossovers.  Whenever  more  targets  are  avail¬ 
able,  more  responses  are  made. 

2.  In  every  fifth  of  the  terrain,  the  number  of  ’’possible"  targets  exceeds  the 
number  "available"  by  an  approximately  constant  amount. 

3.  The  number  of  responses  (objects  designated  as  targets)  made  by  the  no 
choice  limit  group  exceeds  the  number  of  targets  present  by  either  the  possible 
or  available  criteria  m  every  one  of  the  five  intervals. 

4.  In  every  fifth  of  the  terrain  strip  more  responses  are  made  by  groups  having 
a  larger  number  of  total  allowed  choices. 
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The  degree  of  parallelism  of  the  curves  can  be  examined  analytically  by 
calculating  the  product  moment  coefficient  of  correlation  between  number  of 
responses  and  number  of  targets  present.  The  correlation  coefficients  are 
given  in  table  VU.  All  correlations  are  large,  positive,  and  significantly 
(P  .05)  greater  than  zero.  Thus,  the  covariation  of  number  of  responses 
and  number  of  targets  present  is  shown  by  statistics  to  not  be  a  chance  result, 
which  is  also  true  for  the  increase  in  number  of  responses  as  the  total  number 
of  permitted  choices  increases.  The  parallelism  of  the  curves  in  the  figure 
and  the  consequent  high  correlations  indicate  that  the  pattern  of  responding, 
which  may  be  referred  to  as  "pacing, "  merits  further  examination. 


TABLE  Vn 

PRODUCT  MOMENT  CORRELATIONS  BETWEEN  NUMBER  OF 
RESPONSES  PER  SUCCESSIVE  FIFTH  OF  THE  STRIP  OF 
TERRAIN  AND  THE  NUMBER  OF  TARGETS  PRESENT  PER  FIFTH 


TEST  GROUP 

PRODUCT  MOMENT  CORRELATION  COEFFICIENTS 

AVAILABLE*  TARGETS 

POSSIBLE**  TARGETS 

20-cboice 

. 8643*** 

.9754 

40-choice 

.9958 

.9952 

No  limit 

.9518 

.9968 

*  63  targets  in  all  were  judged  prior  to  tests  to  be  available. 

**  50  targets  were  possule  in  that  they  were  found  and  reported  by  one  or 

more  of  the  42  subjects.  These  targets  each  had  a  detection  probability 
equal  to  or  greater  than  .024. 

***  By  one-tailed  tests  of  statistical  significance  all  correlation  coefficients 
in  the  table  are  significantly  greater  than  zero  at  the  .01  level  of  signif¬ 
icance  except  the  one  with  three  asterisks  which  is  significant  at  the  .  05 
level.  Thus,  all  six  relationships  are  greater  than  could  be  expected  by 
chance  alone. 
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To  examine  this  pacing,,  responses  were  summed  up  through  successive 
fifths  of  terrain.  This  summing  smooths  irregularities  and  makes  for  simpler 
comparison  of  pacing  effects.  Table  VUI  gives  the  data,  and  it  is  shown  graph¬ 
ically  in  figure  6.  The  solid  lines  are  least-sqvares  best  fits  to  the  data.  They 
show  what  performance  could  be  expected  if  ail  choices  were  expended  at  a  con¬ 
stant  rate  along  the  terrain.  Note  how  well  the  data  points  fit  the  lines.  It  is 
logical  to  call  the  lines  "patterns  of  pacing. "  The  test  groups  clearly  differ  in 
rate  of  responding  ( slope  of  the  curves)  and  height  of  the  curve?  above  the 
baseline,  but  do  not  differ  in  pattern  or  shape.  The  test  groups  start  out  at 
different  rates  of  responding,  and  the  different  rates  are  maintained  with  little 
fluctuation  when  allowance  is  made  for  differences  in  numbers  of  targets  avail¬ 
able  per  fifth. 


TABLE  Vin 

MEAN  NUMBER  OF  RESPONSES*  ACCUMULATED** 
UP  THROUGH  SUCCESSIVE  FIFTHS  OF  THE  TERRAIN 


MEAN  OF  DETECTED  TARGETS  AND  FALSE  POSITIVES 

Fifths  of 

TEST  GROUP 

_ 

The  Terrain 

20-Choices 

40-Choices 

No- Limit 

First  One 

6.86 

10.29 

20.79 

First  Two 

9.57 

15.65 

33.15 

First  Three 

13.86 

22.51 

47.15 

First  Four 

18.93 

33.37 

66.08 

All  Five 

19.64 

37.73 

76.65 

*  Responses  include  detected  targets  plus  nontargets  designated  as  targets. 

**  The  means  in  this  table  were  obtained  by  summing  group  means  from  table  VI. 

Note  that  there  were  enough  detectable  targets  and  objects  not  distinguishable 
from  targets  by  the  subjects  for  all  individuals  in  the  20-chcice  and  40-choice 
groups  to  very  quickly  use  up  all  or  almost  all  allowed  choices.  The  observed 
patterns  of  pacing  might  not  have  held  if  conditions  had  been  otherwise;  for 
example,  if  the  last  half  of  the  flight  had  contained  only  a  few  possible  targets. 
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FIFTHS  OF  TERRAIN  EXAMINED 

Figure  6.  Summed  Average  Number  of  Responses  Through  the  Course  of  the 
Test  Run  by  Successive  Fifths  of  Terrain. 


That  subjects  in  the  20-choice  group  spread  their  responses  over  the  entire 
path  while  the  40-choice  group  used  20  choices  by  the  time  they  were  halfway 
along  the  flight  path  is  significant.  Subjects  were  not  told  to  spread  (or  not  to 
spread)  their  responses  over  the  entire  flight  path.  How  strongly  the  results 
are  affected  by  the  target  distribution  and  how  much  is  attributable  to  delibe¬ 
rate  attempts  to  pace  is  unknown.  However,  the  large  increase  in  accuracy  as 
the  permitted  number  of  target  choices  decreased  (a  point  discussed  elsewhere 
in  this  report)  indicated  that  subjects  were  more  cautious  when  they  believed 
that  more  caution  in  target  selection  could  be  profitably  exercised.  When  they 
had  a  larger  number  of  allowed  choices  (available  weapons),  they  acted  as  if 
greater  recklessness  were  justified,  presumably  to  avoid  ending  the  mission 
with  several  unexpended  weapons. 

The  generality  of  the  finding  of  uniformity  (with  correction  for  target 
availability)  along  the  entire  strip  of  terrain  of  rate  of  choosing  targets  is 
unknown.  Also,  the  generality  of  the  high  correlations  between  number  of 
responses  and  number  of  available  targets  is  unknown.  The  authors  believe 
that  both  findings  will  hold  up  in  subsequent  research  that  further  examines 
the  variation  in  distribution  of  targets  over  the  imaged  terrain.  Such  research 
should  be  done  with  various  sets  of  instructions. 
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DISTRIBUTION  OF  TARGET  DETECTIONS  OVER  THE  TERRAIN 

The  previous  section  showed  that  total  number  of  responses,  including  both 
detections  and  false  positives,  accumulated  over  the  test  session  at  a  highly 
uniform  rate,  and  these  rates  increased  with  the  number  of  allowed  choices. 
When  detected  targets  alone  are  considered,  exactly  the  same  conclusions  can 
be  drawn.  This  is  true  when  detections  by  either  successive  groups  of  ten 
available  targets  or  by  successive  fifths  of  terrain.  This  may  be  readily  seen 
from  examination  of  figure  7,  which  is  plotted  from  the  data  in  tables  IX  and  X. 
The  two  sets  of  curves,  solid  and  dashed,  are  almost  identical. 


CONSECUTIVE  FIFTH  OF  TERRAIN  (DASHED  LINES) 

1ST  I  ST  2  IST3  I  ST 4  IST5 


Figure  7.  Rate  of  Accumulation  of  Target  Detections  as  Shown  by  Percentage 
of  Available  Targets  Found  up  Through  Consecutive  Tenths  of  the 
Available  Targets  and  up  Through  Consecutive  Fifths  of  the  Strip 
oi'  Terrain. 


VARIATION  IN  ACCURACY  OVER  THE  COURSE  OF  THE  SIMULATED  MISSION 

’reviously,  it  was  shown  that  over  the  entire  flight  path,  i.e.,  for  the 
simulated  mission  taken  as  a  wl  ole,  higher  accuracies  were  obtained  when 
fewer  target  cho.ces  were  permitted.  The  flight  path  may  be  divided  into  sec¬ 
tions  on  the  basis  of  either  length  or  of  number  of  targets  to  examine  accuracy 
as  a  function  of  degree  of  completion  of  the  mission.  Target  selection  behavior 


TABLE  IX 


MEAN  PERCENTAGE*  AND  CUMULATIVE  PERCENTAGE  OF  TARGETS 
DETECTED  IN  EACH  CONSECUTIVE  FIFTH  OF  THE  TERRAIN 


TEST  GROUP 

Fifth  of 

Available 

20-Choice 

40-Choice 

No-Limit 

Terrain 

Targets 

Mean% 

Sum98** 

Mean% 

Sum% 

Mean% 

Sum% 

First 

14 

9.43 

9.43 

13.00 

13.00 

14.71 

14.71 

Second 

6 

3.00 

12.43 

6.29 

19.28 

7.57 

22.29 

Third 

9 

6.14 

18.57 

8.00 

27.28 

10.43 

32.71 

Fourth 

16 

9.57 

28.14 

15.00 

42.28 

17.57 

50. 29 

Fifth 

5 

1.29 

29.43 

4.29 

46.57 

6.71 

57.00 

Sum 

50 

29.43 

- - 

1 

1 

46.58 

1 

1 

56.99 

_ 

1 

1 

*  Based  on  the  50  targets  that  were  each  detected  by  one  or  more  of  the  42  test 
subjects,  not  on  the  numbers  in  individual  fifths  of  terrain. 


**  Sum  %  is  the  percentage  of  the  total  50  targets  detected  up  through  the  fifth  of  the 
terrain  (3rd,  etc.) 

TABLE  X 

MEAN  PERCENTAGE  AND  CUMULATIVE  PERCENTAGE  OF 
THE  TOTAL  OF  50  TARGETS  THAT  WERE  DETECTED  IN 
EACH  CONSECUTIVE  GROUP  OF  10  DETECTABLE*  TARGETS 


TEST  GROUP 

No- Limit  j 

Targets 

Sum% 

Sum% 

Mean% 

Sum% 

1-10 

6.857 

6.857 

9.143 

9. 143 

10.2S6 

10.286 

11-20 

5.  571 

12.428 

10.143 

19.286 

12.000 

22.286 

21-30 

6.143 

18.571 

9.000 

28.286 

>1.286 

33.572 

31-40 

8.000 

26.571 

11.571 

39.857 

46.572 

1  41-50 

L - 

2.857 

29.428 

6.714 

46.571 

10.429 

!  Sum 
l _ 

29.428 

— 

46.571 

— 

57.001 

*  Detectable  in  that  one  or  more  of  the  42  subjects  recorded  them  as  targets. 
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could  change  during  the  mission.  Teat  subjects  could  become  more  cautious 
or  less  cautious  in  selecting  targets,  depending  upon  how  rapidly  they  decide 
that  they  are  using  up  their  allowed  number  of  choices. (or  strike  weapons) 
Subjects  may,  early  in  a  mission,  develop  ex  ectancies  about  frequency  of 
occurrence  of  targets  that  will  influence  their  selection  behavior.  This  was 
discussed  at  length  in  the  section  on  Distribution  of  Responses  Over  the  Terrain. 

Target  visibility,  or  ease  and  certainty  of  finding  and  recognizing  targets, 
will  normally  vary  during  the  course  of  a  simulated  mission.  This  is  a  source 
of  variability  in  accuracy  in  addition  to  accuracy  changes  that  are  attributable 
to  changes  in  the  degree  of  caution  or  selectivity  of  the  observer.  The  numeri¬ 
cal  value  of  the  accuracy  is  influenced  by  many  variables  that  were  unmeasured 
and/or  uncontrolled.  Thus,  it  is  of  interest  to  compare  the  accuracies  of  the 
various  test  groups. 

The  average  accuracy  for  each  choice  group  for  successive  fifths  of  the 
terrain  is  given  in  table  XI  and  is  plotted  in  figure  8.  Although  there  are  fluc¬ 
tuations,  the  accuracy  curves  for  the  no-limit  and  the  40-choice  groups  clearly 
do  not  exhibit  any  overall  trend.  For  the  20 -choice  group,  accuracy  increases 
with  the  duration  of  the  simulated  mission.  The  linear  component  of  this  posi¬ 
tive  trend  is  shown  to  be  a  non-chance  occurrence  if  the  correlation  between 
accuracy  and  fifth  of  the  terrain  is  significantly  different  from  zero.  The  pro¬ 
duct  moment  correlation  was  +.  849.  It  became  +.  862  when  normalizing  require¬ 
ments  were  satisfied  by  subjecting  the  dat$  to  an  arc  sine  transformation. 

Since  a  correlation  of  only  .  805  is  required  for  statistical  significance  at  the 
.05  level,  the  obtained  trend  can  not  be  attributed  to  chance. 


TABLE  XI 

AVERAGE  ACCURACY  OF  RESPONSES  PER  FIFTH  OF 
THE  STRIP  OF  TERRAIN  FOR  THE  THREE  TEST  GROUPS 


EXPERIMENTAL 

GROUP 

FIFTH  OF  THE  STRIP  OF  TERRAIN 

OVERALL 

FIRST 

SECOND 

THIRD 

FOURTH 

FIFTH 

MEAN 

20-Choice 

.6957 

.5774 

.7429 

.9531 

.9523 

.7485 

40-Choice 

.6614 

.6083 

.6033 

.7143 

.  o581 

.6208 

No- Limit 

.3827 

.3381 

.4206 

.4837 

.3049 

.3849 
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Figure  8.  Average  Accuracy  by  Fifths  of  Terrain  for  the  Three  Test  Groups. 


The  40-choice  and  the  no-limit  groups  did  not  exhibit  a  trend  in  accuracy 
with  duration  of  the  simulated  mission;  hence,  it  is  likely  that  average  target 
visibility  did  not  change  appreciably  during  the  flight.  Possibly  the  40-choice 
and  no-limit  groups  became  less  cautious  as  the  experiment  progressed,  and 
if  average  visibility  of  targets  also  increased,  then  accuracy  might  not  ap¬ 
preciably  alter  during  the  test  session.  These  possibilities  are  deemed  un¬ 
likely,  For  these  two  groups,  it  is  also  unlikely  that  caution  increased  but 
target  visibility  decreased,  so  that  no  trend  occurred  in  accuracy.  It  is  con¬ 
cluded  that  the  observers  with  only  20  allowed  choices  tended  to  become  more 
cautious  as  the  duration  of  the  flight  increased.  Presumably,  expectancies 
built  up  during  the  early  parts  of  the  simulated  mission  led  them  to  believe 
that  they  cculd  afford  to  be  more  selective  in  what  they  designate  as  targets 
and  still  be  able  to  wisely  expend  all  of  their  weapons. 

It  appears  reasonable  to  expect  that  accuracy  will  increase  as  the  number 
of  unused  choices  decreases.  To  examine  this  possibility  for  the  20-choice 
group,  accuracy  was  plotted  against  the  number  of  choices  that  had  been  made 
{see  fig  9) .  No  strong  trend  is  apparent  in  this  figure,  but  the  average  of  the 
last  third  of  the  curve  is  well  above  the  average  for  the  first  third.  Thus,  a 
weak  trend  may  be  present.  To  examine  this  possibility,  the  product  moment 
correlation  between  accuracy  and  number  of  choices  made,  using  untransformed 
scores,  was  calculated.  It  is  +.477,  which  is  larger  than  the  .378  value 
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N= CHOICE  NUMBER  (FROM  FIRST  TO  TWENTIEH) 


Figure  9.  Average  Accuracy  from  the  First  to  Twentieth  Choice  for  the 
20-Choice  Group. 

required  for  statistical  significance  at  the  .  05  level.  It  s  concluded  that  a 
positive  but  not  high  degree  of  relationship  exists  between  accuracy  and  number 
of  choices  that  have  been  made.  This  finding  is  consistent  with  the  positive 
relationship  found  for  the  20-choice  group  between  accuracy  and  mission  dura¬ 
tion  as  measured  by  successive  fifths  of  terrain. 

SCREEN  TRAVEL 

The  distance  that  the  image  of  an  object  moves  down  the  display  screen 
before  the  observers  points  to  it  and  labels  it  according  to  target  type,  is 
called  screen  travel.  The  average  screen  travel  for  the  three  test  groups  is 
shown  in  table  XII.  Bartlett's  test  for  homogeneity  of  K  variances  revealed 
no  statistically  significant  heterogeneity  of  variance  in  the  screen  travel  of 
detected  targets  (correct  choices) ,  or  in  the  screen  travel  of  false  positives 
(incorrect  choices) .  The  hypothesis  that  the  populations  represented  by  the 
correct  choice  data  have  equal  variaiices  is  acceptable,  as  is  the  hypothesis 
that  population  variances  are  equal  for  the  screen  travel  of  false  positives. 

The  average  distances  (means)  traveled  by  real  targets  prior  to  a  response, 
as  shown  in  the  table,  were  similar  for  all  three  of  the  choice  groups.  The 
analysis  of  variance  of  mean  distance  traveled  is  given  in  table  xm.  It  shows 
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TABLE  XII 


AVERAGE  DISTANCE*  TRAVELED  BY  TARGET  IMAGES  BETWEEN 
THEIR  INITIAL  APPEARANCE  ON  THE  DISPLAY  A  D  THEIR  DETECTION 


Test  Group 

Mean 

Standard  Deviation 

20-Choice 

5.29 

.86 

40-Choice 

5.48 

.94 

No- Limit 

4.95 

1.03 

*  Distances  are  in  inches  on  the  screen. 
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TABLE  XIH 

ANALYSIS  OF  VARIANCE  FOR  DISTANCE  TRAVELED  BY  TARGETS 

PRIOR  TO  DETECTION 


Source  of  Variance 

ra 

Mean  Square 

F 

Meaning 

Weapon  load 

2 

0.6414 

0.7166 

The  number  of  choices 

available  had  no  signi- 

Within  groups  variation 

39 

0.8951 

ficant  effect  upon  the 

average  distance  trav-  j 

Total 

41 

eled  by  targets  prior  | 

_ 

to  their  detection.  | 

_ 1 

Note:  Statistical  significance  at  the  .05  level  was  not  attained. 

■ 
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that  distances  (reaction  times)  did  not  differ  significantly  for  the  three  choice 
groups.  The  mean  distances  were  subjected  to  multiple  comparisons  by  use  of 
Duncan's  Test  (NMRT) .  The  same  analysis  was  then  performed  on  the  mean 
distances  traveled  by  false  positives.  In  neither  case  was  statistical  signifi¬ 
cance  achieved  at  the  .05  level.  It  is  concluded  that  the  number  of  allowed 
choices  (responses)  has  no  significant  effect  upon  the  average  distance  traveled 
on  the  display,  prior  to  detection,  by  the  images  of  either  real  targets  or  of 
objects  mistaken  for  targets.  In  brief,  response  to  targets  is  equally  rapid  for 
the  three  test  groups,  as  is  the  case  for  false  positives. 

For  all  three  of  the  test  groups  the  majority  of  the  targets  that  are  found 
are  detected  while  still  on  the  upper  half  of  the  display  screen.  This  is  clearly 
shown  in  figure  10. 


Figure  10.  Percentage  of  Available  Targets  That  Were  Detected  on  the  Upper 
and  Lower  Halves  of  the  Display  Screen.  In  Both  Cases  Fifty 
Targets  Were  Available. 
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DETECTABILITY  OF  DIFFERENT  TYPES  OF  TARGETS 


The  general  rule  that  the  detectability  of  targets  varies  with  the  type  of 
target  applies  to  the  performance  of  observers  using  a  side-looking  radar 
sensor.  Table  XIV  lists  the  numbers  of  targets  detected  and  the  means  and 
percentages  of  targets  detected  for  the  various  types  of  targets.  Figure  11 
shows  the  results:  For  all  four  types  of  targets,  the  percentage  detected  in¬ 
creases  as  the  number  of  allowed  target  choices  increases.  Note  that  perfor¬ 
mance  against  tank  farms  was  consistently  low  relative  to  performance  against 
other  types  of  targets.  Even  witn  no  limit  placed  upon  the  number  of  allowed 
target  choices,  only  19, 1  percent  of  tank  farms  were  detected.  Except  for  the 
20-choice  group,  percentage  of  targets  detected  was  highest  for  railroad  yards, 
reaching  66. 8% for  the  no-limit  group.  Note  the  cross-over  in  figure  11  in  the 
curve  for  airfields,  and  the  slow  increase  in  detections  for  airfields  and  tank 
farms,  as  compared  to  railroad  yards  and  dan's,  with  increase  in  the  number 
of  target  cnoices  that  were  allowed. 


TABLE  XIV 

TARGET  DETECTION  FOR  THE  FOUR  TYPES  OF  TARGETS 


Target 

Type 

Targets 

Present 

TEST  GROUP 

20 

-CHOICES 

40-CHOICES 

NO  LIMIT 

N+ 

Mean 

N+ 

Mean 

% 

N+ 

Mean 

L 

% 

Airfield 

I  I 

72 

5.143 

34.29 

87 

6.214 

41.43 

98 

46.67 

Dam 

46 

3.286 

20.54 

101 

7.214 

45.09 

122 

8.714 

54. 46 

RR  Yard 

61 

4.357 

31.12 

102 

7.286 

131 

9.357 

66.84 

Tank  Farm 

1.928 

10.71 

36 

2.571 

14.29 

48 

3.429 

■■■■ 

I  i  ■ 

Sum 

63 

206 

m 

— 

326 

— 

— 

399 

— 

Mean 

1 

1 

14.714 

S 

— 

23.286 

— 

— 

28.500 

fifl 

— 

+N  is  number  of  targets  detected  by  the  entire  14  subjects  in  the  test  group. 

*  Percentages  are  based  on  the  63  targets  determined,  prior  to  testing  subjects 
have  detectable  and  recognizable  target  signatures.  If  percentage  values  in  the 
table  are  multiplied  by  63/50,  i.e.  by  1.26,  the.  results  will  be  percentage  detec¬ 
tion  against  the  50  targets  found  by  one  or  more  of  tne  42  subjects. 
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20  40  NO  LIMIT 

NUMBER  OF  ALLOWED  TARGET  CHOICES 

Figure  11.  Percentage  of  Targets  of  Different  Types  Detected. 


ACCURACY  FOR  DIFFERENT  TYPES  OF  TARGETS 

In  an  earlier  section  of  this  paper,  overall  accuracy,  where  performance 
against  all  types  of  targets  was  pooled,  was  found  to  rapidly  decrease  as  number 
of  target  choices  increased.  What  happens  for  different  types  of  targets  should 
be  examined  for  more  insight  into  the  overall  decrease  in  accuracy.  For  example, 
it  would  be  important  to  know  whether  or  not  the  accuracy  loss  could  be  attri¬ 
buted  to  only  one  or  two  types  of  targets.  The  accuracy  data  given  in  table  XV 
is  plotted  in  figure  12.  Note  that  with  a  slight  exception  for  dams,  the  rate  of 
decrease  of  proportion  of  responses  that  were  correct,  as  showm  by  the  steepness 
or  siope  of  the  curves,  is  about  the  same  for  the  different  types  of  targets. 
Apparently,  no  one  type  of  target  is  particularly  responsible  for  the  large  in¬ 
crease  in  errors  that  occurs  wher.  more  target  choices  are  permitted.  In  figure  12, 
tne  big  drop  in  accuracy  over  the  three  test  conditions  occurs  in  going  from  the 
4 j -choice  to  the  no-limit  condition. 

Table  XV  and  figure  12  show  that  accuracy  is  highest  for  airfields:  from  .57 
(with  20-choices)  to  .70  (with  no  limit  on  number  of  choices)  of  objects  called 
airfields  by  the  observers  are  truly  airfields.  Clearly,  few  objects  of  other 
types  are  mistaken  for  airfields.  This  is  in  sharp  contrast  to  tank  farms  where 
accuracy  was  lowest.  With  proportion  of  responses  that  were  correct  of  .  54 
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to  .  17  for  the  extreme  conditions,  an  object  identified  as  a  tank  farm  is 
probably  something  else.  When  numbers  of  choices  were  unlimited,  only 
airfield  and  dam  responses  (designations)  were  more  likely  to  be  correct 
than  incorrect.  Note  that  performance  on  tank  farms  was  lowest  for  the  no- 
lirait  choice  group  as  measured  by  either  accuracy  or  percentage  of  targets 
detected.  Also,  railroad  yards,  while  high  on  percentage  detected,  were 
relatively  low  on  response  accuracy:  Most  of  them  were  found,  but  other 
objects  were  often  mistaken  for  them. 


TABLE  XV 

ACCURACY  BY  TARGET  TYPE  FOR  THE  THREE  TEST  GROUPS 


TEST  GROUP 

TYPE  OF  TARGET 

AIRFIELDS 

DAMS 

RR  YARDS 

TANK  FARMS 

20-Choice 

.9673 

.8151 

. 

.7120 

.  5429 

40-Choice 

.9123 

.6301 

6200 

.4158 

No- Limit 

.7005 

.5784 

.4003 

.1678 

i.Or 

.9 

£  .81- 
j— 

20 
olu  7 
r;  e  ■' 

gS  .6 

o  UJ 

ss 

ig 

<x> 

=>UJ 
Olu 

<z 

<(/l 
UjU 


.5 
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20  40  NO  L*MIT 

NUMBER  OF  AVAILABLE  TARGET  CHOICES 


Figure  12.  Accuracy  Versus  Number  of  Available  Target  Choices  for  the 
Four  Different  Types  of  Targets. 
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EFFECT  OF  TEST  CONDITIONS  ON  DIFFERENT  TYPES  OF  TARGETS 

Figure  13  and  table  XVI  show  the  number  of  different  targets  of  each  type 
reported  by  one  or  more  individuals.  With  each  increase  in  number  of  choices 
allowed,  there  was  an  increase  in  the  number  of  different  targets  of  each  type 
reported.  Clearly,  this  increase  was  in  individual  targets  not  reported  by 
anyone  in  the  more  restricted  test  groups.  When  all  types  of  targets  were 
combined,  the  unlimited  choice  group  reported  44%  more  different  targets  than 
did  the  20-choice  group. 

Figure  13  shows  that  when  allowed  number  of  choices  increases  greatiy, 
there  is  only  a  small  increase  in  the  number  of  reported  different  railroad 
yards  and  airfields.  However,  there  in  a  large  increase  in  different  dams  and 
tank  farms.  Thus,  restricting  number  of  allowed  choices  had  a  differential 
influence  upon  the  detection  of  the  different  types  of  targets.  The  targets  not 
reported  at  all  in  the  two  limited-choice  groups  were  responded  to  infrequently 
in  the  unlimited  choice  group.  They  were  likely  the  targets  that  were  the  most 
difficult  to  find  and/or  were  less  obviously  targets  when  they  were  found. 


20  40 

NUMBER  Of  ALLOWED  TARGET  CHOCES 


Figure  13.  Number  of  Different  Targets  of  Each  Type  Detected  With 
Different  Numbers  of  Allowed  Target  Choices. 
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TABLE  XVI 


NUMBER  OF  DIFFERENT  TARGETS  OF  EACH  TYPE  DETECTED 


Target  Type 

NUMBER  DETECTED  BY  THE  14  TEST 
SUBJECTS  IN  EACH  GROUP 

Different 

Targets 

present* 

TEST  GROUP 

20-Choices 

40-Choices 

No-Limit 

Airfield 

8 

I 

9  1 

10 

15 

Dam 

10 

15 

14 

16 

RR  Yard 

13 

14 

14 

14 

Tank  Farm 

3 

5 

11 

18 

Sum 

34 

43 

4S 

63 

Mean/Subject 

2.43 

3.07 

3.50 

*  As  determined  by  image  truth  prior  to  testing  subjects,  i.e. ,  prejudged 
as  being  detectable. 
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SECTION  IV 


RECOM  MENDATIONS 


An  unacceptably  high  proportion  of  false -positives  has  been  found  in 
numerous  studies  by  the  present  authors  and  by  other  investigators.  These 
studies  have  presented  images  of  the  real  world  to  observer  and  have  usually 
placed  little  or  no  restrictions  upon  th?  number  of  targets  that  observers 
could  designate.  Obviously  some  errors  are  due  to  marginally  resolved 
target  and  to  nontarget  images.  The  present  study  demonstrates  that  limit¬ 
ing  the  number  of  targets  that  may  be  chosen  can  greatly  reduce  the  number 
and  proportion  of  nontargets  that  are  reported  as  targets.  This  reduction 
is  understandably  achieved  at  a  cost  in  the  percentage  of  actual  targets  de¬ 
tected.  Despite  this,  the  study  shows  that  response  restrictions  could  make 
useful  some  rapid  reconnaissance  and  reconnaissance/strike  systems  that 
may  have  appeared,  on  the  basis  of  prior  studies  and  analyses,  to  be  feasible 
because  of  high  false-positive  rates.  Thus,  the  following  recommendations 
are  made: 

1 .  Since  the  false-positive  rates  in  studies  without  realistic  limits  on  number 
of  responses  allowed  are  misleading.  These  results  must  be  interpreted  in 
the  light  of  the  findings  of  the  present  study. 

2.  Reconnaissance  and  reconnaissance/strike  systems  studies  should  impose 
realistic  limitations  on  number  of  allowed  target  choices. 

3.  The  realizable  potentials  for  target  finding  systems  that  can  result  from 

use  of  appropriate  instructions  and  orientation,  from  efficient  utilization  of 
confidence  judgment  information,  and  from  optimum  restrictions  upon  the 
number  of  permitted  target  choices  is  unknown.  More  information  is 
necessary  on  the  utility-of-choice  restrictions  versus  confidence  judgments. 
This  can  be  obtained  in  studies  using  the  same  stimulus  materials.  Trade¬ 
off  information  from  such  studies  is  needed  by  training  and  using  organizations 
dealing  with  images  of  the  real  world.  » 
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APPENDIX 


INSTRUCTIONS  TO  THE  TEST  SUBJECTS 

INSTRUCTIONS 


This  mission  flies  over  hostile  territory  at  a  speed  of  1320  knots  for  27 
minutes.  The  picture,  covering  a  14  by  14-ineh  screen,  depicts  a  strip  of 
ground  25  nautical  miles  wide.  The  image  moves  down  the  screen  at  about 
12  inches  per  minute.  The  targets,  as  shown  by  the  illuminated  push  buttons, 
are: 


Airfields 

Dams 

Railroad  Yards 
Tank  Farms 

When  you  find  a  target,  depress  the  proper  target  name  switch,  place  the 
tip  of  the  illuminated  pointer  on  the  target,  and  press  the  "record"  switch  to 
allow  the  data  camera  to  record  your  response.  Be  rare  at  that  time  that  your 
head  or  shoulder  does  not  block  the  view  of  the  camera.  Do  not  be  too  cautious 
to  attack  targets,  but  he  careful  not  to  waste  weapons  on  nontarget  objects, 
i.e. ,  before  launching  a  weapon,  be  fairly  sure  that  you  are  aiming  a.  a  target. 
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INSTRUCTIONS 


This  mission  flies  ovei  hostile  territory,  at  a  speed  of  1320  knots  for 
27  minutes.  The  picture,  or.  a  J4  h  14-inch  screen,  depicts  a  strip  of 
ground  25  nautical  miles  wide.  The  image  moves  down  the  screen  at  about 
12  inches  per  minute.  The  targets  as  shown  by  the  illuminated  push  buttons, 
are: 


Airfields 

Dams 

Railroad  Yards 
Tank  Farms 

Your  aircraft  carries  20  bombs  (or  missiles) .  Your  task  is  to  make  the 
best  use  of  them:  Do  not  be  too  cautious  to  attack  targets,  but  be  careful  not 
to  waste  weapons  on  nontarget  objects,  i.e. ,  before  launching  a  weapon,  be 
fairly  sure  that  you  are  aiming  at  a  target.  Bringing  a  weapon  back  to  home 
base  is  preferable  to  wasting  it  on  a  nontarget.  You  will  not  be  told  how  many 
targets  are  on  the  filmstrip. 

A  clock  and  a  counter  on  the  console  always  display  the  time  and  the  num¬ 
ber  of  weapons  remaining;  check  both  of  them  occasionally,  as  well  as  the  list 
of  target  push  buttons,  to  keep  track  of  target  types,  times  and  munitions. 

When  a  target  is  found,  depress  the  proper  target  name  switch,  place  the 
illuminated  pointer’s  tip  on  the  target,  and  push  the  "record"  switch  to  record 
the  target  choice.  Be  sure  at  that  time  that  your  head  or  shoulder  does  not 
block  the  view  of  the  camera. 
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